Light field display control method and apparatus, and light field display device

ABSTRACT

Embodiments of the present application disclose a light field display control method and apparatus and a light field display device. The light field display control method comprises: determining at least one depth distribution sub-region of content according to a display depth of field (DoF) range of a light field display device and depth distribution information of the content, wherein each depth distribution sub-region of the at least one depth distribution sub-region is located outside the display DoF range; and tilting a first lenslet with respect to an original plane of a lenslet array of the light field display device according to at least the display DoF range of the light field display device and the depth distribution sub-region, wherein the first lenslet is a lenslet that is in the lenslet array of the light field display device and affects display of a first object, and the first object is a part, which is located in the depth distribution sub-region, of the content. The present application can improve display quality of an object, which is located in a partial depth distribution sub-region, of content to be displayed or content being displayed.

CROSS REFERENCE TO RELATED APPLICATION

The present application is based on and claims the benefit of priorityto Chinese Application No. 201511031332.7, filed on Dec. 31, 2015, theentire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present application relates to the field of light field displaytechnologies, and in particular, to a light field display control methodand apparatus, and a light field display device.

BACKGROUND

A light field display technology was proposed early in the 20^(th)century, and two representative implementation manners of the lightfield display technology are: a lenslet array-based implementationmanner proposed by Lippmann in 1908 and a parallax barriers-basedimplementation manner proposed by Ives in 1903. In recent years, asconsumer electronics impose diversified requirements on display devices,the light field display technology is applied to different scenarios anddevices, such as a 3D monitor, a wearable device, and a light fielddisplay device for vision correction. At present, the computing powerand display resolution of electronic devices gradually match with ahardware resource requirement of the light field display technology,which provides desirable support for the promotion and application ofthe light field display technology.

By means of a hardware structure similar to that of a conventional lightfield display technology, the light field display technology canimplement relatively flexible display effects, such as light field 3Ddisplay, light field projection display, light field near-to-eye displayon a wearable device, and vision correction by means of light fielddisplay.

SUMMARY

The following provides a brief summary about the present application, soas to provide basic comprehension about some aspects of the presentapplication. It should be understood that, the summary is not anexhaustive summary of the present application. It is neither intended todetermine the key part or important part of the present application norintended to limit the scope of the present application. The objectivethereof is merely to provide some concepts in a simplified form, toserve as a prelude for subsequent more detailed descriptions.

The present application provides a light field display control methodand apparatus and a light field display device.

According to a first aspect, an embodiment of the present applicationprovides a light field display control method, comprising:

determining at least one depth distribution sub-region of contentaccording to a display depth of field (DoF) range of a light fielddisplay device and depth distribution information of the content,wherein each depth distribution sub-region of the at least one depthdistribution sub-region is located outside the display DoF range; and

tilting a first lenslet with respect to an original plane of a lensletarray of the light field display device according to at least thedisplay DoF range of the light field display device and the depthdistribution sub-region, wherein the first lenslet is a lenslet that isin the lenslet array of the light field display device and affectsdisplay of a first object, and the first object is a part, which islocated in the depth distribution sub-region, of the content.

According to a second aspect, an embodiment of the present applicationfurther provides a light field display control apparatus, comprising:

a depth distribution sub-region determining module, configured todetermine at least one depth distribution sub-region of contentaccording to a display DoF range of a light field display device anddepth distribution information of the content, wherein each depthdistribution sub-region of the at least one depth distributionsub-region is located outside the display DoF range; and

a tilt control module, configured to tilt a first lenslet with respectto an original plane of a lenslet array of the light field displaydevice according to at least the display DoF range of the light fielddisplay device and the depth distribution sub-region, wherein the firstlenslet is a lenslet that is in the lenslet array of the light fielddisplay device and affects display of a first object, and the firstobject is a part, which is located in the depth distribution sub-region,of the content.

According to a third aspect, an embodiment of the present applicationprovides another light field display control apparatus, comprising:

a processor, a communications interface, a memory, and a communicationsbus, wherein the processor, the communications interface, and the memorycommunicate with each other by means of the communications bus;

the memory is configured to store at least one instruction, and theinstruction causes the processor to perform the following operations:

determining at least one depth distribution sub-region of contentaccording to a display DoF range of a light field display device anddepth distribution information of the content, wherein each depthdistribution sub-region of the at least one depth distributionsub-region is located outside the display DoF range; and

tilting a first lenslet with respect to an original plane of a lensletarray of the light field display device according to at least thedisplay DoF range of the light field display device and the depthdistribution sub-region, wherein the first lenslet is a lenslet that isin the lenslet array of the light field display device and affectsdisplay of a first object, and the first object is a part, which islocated in the depth distribution sub-region, of the content.

According to a fourth aspect, an embodiment of the present applicationprovides a light field display device, comprising:

a monitor;

a lenslet array, wherein the lenslet array comprises multiple tiltablelenslets distributed in the array; and

any light field display control apparatus described, wherein the lightfield display control apparatus is connected to the monitor and thelenslet array.

In the technical solutions provided by the embodiments of the presentapplication, the first lenslet is tilted with respect to the originalplane of the lenslet array (wherein the original plane is a plane inwhich the lenslet array is located when all lenslets of the lensletarray are in a non-tilted state) according to at least a current displayDoF range of the light field display device and a depth distributionsub-region on which image quality control needs to be performed, tochange a display image distance of the first object in space by means ofcontrol over the tilt of the first lenslet, so that an average displayimage distance of a virtual image which is formed after the first objectis displayed by the first lenslet is distributed in the current displayDoF range of the light field display device as far as possible, therebyimproving display quality of the first object displayed by the firstlenslet. In this solution, on the basis of clear imaging in an originaldisplay DoF range, a control means of tilting a lenslet is fullyutilized to improve display quality of an object in at least one depthdistribution sub-region outside the display DoF range, achieving aneffect equivalent to that a user can clearly see an object which is in adepth range larger than the original display DoF range; moreover, in thedisplay quality adjustment process, it is unnecessary to perform complexcomputation on source content of a light field, and by adjusting a focallength of a corresponding lenslet, display quality of an object outsidethe original display DoF range can be adjusted. The solution is simpleand easy to control.

Through the following detailed description of optional embodiments ofthe present application with reference to the accompanying drawings,these and other advantages of the present application will be clearer.

BRIEF DESCRIPTION OF THE DRAWINGS

The present application may be better comprehended with reference to thefollowing description provided in combination with the accompanyingdrawings, wherein same or similar reference numerals are used in all theaccompanying drawings to represent same or similar components. Theaccompanying drawings together with the following detailed description,which are incorporated in the specification and form a part of thespecification, are used to further illustrate the optional embodimentsof the present application and explain the principle and advantage ofthe present application. In the accompanying drawings:

FIG. 1 is a flowchart of a light field display control method accordingto an embodiment of the present application;

FIG. 2a illustrates a first example of a light field display device andlight field display thereof according to an embodiment of the presentapplication;

FIG. 2b illustrates a second example of a light field display device andlight field display thereof according to an embodiment of the presentapplication;

FIG. 2c illustrates an example of determining a first lenslet accordingto an embodiment of the present application;

FIG. 3a illustrates a continuous depth distribution example of a depthdistribution relationship between a depth distribution sub-region and aDoF range according to an embodiment of the present application;

FIG. 3b illustrates a discontinuous depth distribution example of adepth distribution relationship between a depth distribution sub-regionand a DoF range according to an embodiment of the present application;

FIG. 3c illustrates a continuous depth distribution example and adiscontinuous depth distribution example of a depth distributionrelationship between depth distribution sub-regions and a DoF rangeaccording to an embodiment of the present application;

FIG. 4 is a logic block diagram of a light field display controlapparatus according to an embodiment of the present application;

FIG. 5 is a logic block diagram of a tilt control module according to anembodiment of the present application;

FIG. 6 is a logic block diagram of an expected display depth informationacquiring sub-module according to an embodiment of the presentapplication;

FIG. 7 is a logic block diagram of a light field display controlapparatus according to an embodiment of the present application;

FIG. 8 is a logic block diagram of a light field display controlapparatus according to an embodiment of the present application; and

FIG. 9 is a schematic structural diagram of a light field display deviceaccording to an embodiment of the present application.

A person skilled in the art should understand that, elements in theaccompanying drawings are merely shown for the purpose of simplicity andclarity, but are not necessarily drawn proportionally. For example,sizes of some elements in the accompanying drawings may be enlargedrelative to other elements, to help understand the embodiments of thepresent application.

DETAILED DESCRIPTION

Exemplary embodiments of the present application are described in detailin the following through the accompanying drawings. For clarity andsimplicity, not all features of actual implementation manners aredescribed in the specification. However, it should be understood that ina process of developing any such actual embodiment, many decisionsspecific to the implementation manner need to be made to implementspecific objectives of developers, for example, meeting restrictiveconditions related to a system and service, and the restrictiveconditions may vary according to different implementation manners.Moreover, it should be further understood that, although the developmentwork may be very complex and time consuming, the development work ismerely a routine task for a person skilled in the art who is benefitedfrom the content of the present disclosure.

Here, it should be further noted that, to prevent the presentapplication from being blurred by unnecessary details, merely apparatusstructures and/or processing steps closely related to the solutions ofthe present application are described in the accompanying drawings andthe specification, and expressions and descriptions about components andprocessing that are less related to the present application and known bya person of ordinary skill in the art are omitted.

Specific implementation manners of the present application are furtherdescribed in detail with reference to the accompanying drawings (samereference numerals in several accompanying drawings represent sameelements) and embodiments. The following embodiments are used todescribe the present application, but are not intended to limit thescope of the present application.

A person skilled in the art should understand that terms such as “first”and “second” in the present application are merely used to distinguishdifferent steps, devices, modules or the like, and the terms neitherrepresent any specific technical meanings nor indicate necessary logicorders between them.

FIG. 1 is a flowchart of a light field display control method accordingto an embodiment of the present application. The light field displaycontrol method provided in an embodiment of the present application maybe executed by a light field display control apparatus, and the lightfield display control apparatus may execute the light field displaycontrol method in application programs that comprise but not limited to:image presentation, video playback, and the like, to perform imagedisplay control. A device manifestation form of the light field displaycontrol apparatus is not limited. For example, the light field displaycontrol apparatus may be an independent component, and the component isin coordination and communication with a light field display device; orthe light field display control apparatus may be integrated in a lightfield display device as a functional module, and the light field displaydevice may comprise but not limited to: an electronic device having alight field display capability. For example, the light field displaydevice may comprise but not limited to: a near-to-eye light fielddisplay device, and the near-to-eye light field display device maycomprise but not limited to: a smart helmet, smart glasses, and thelike. Specifically, as shown in FIG. 1, a light field display controlmethod provided in an embodiment of the present application comprises:

S101: Determine at least one depth distribution sub-region of contentaccording to a display DoF range of a light field display device anddepth distribution information of the content, wherein each depthdistribution sub-region of the at least one depth distributionsub-region is located outside the display DoF range.

The light field display device comprises a monitor and a lenslet arraythat are arranged close to each other, as shown in FIG. 2a . The monitormay be a continuous monitor or the monitor may be a screen formed bymultiple connected display units that are distributed in an array,wherein the monitor may be divided into multiple display sub-regionswhen displaying content, each display sub-region comprises multipledisplay pixels distributed in an array, each display sub-region candisplay a sub-image, and the sub-image is a part of an image of thecontent. The lenslet array may also be referred to as a micro-lensarray, and comprises multiple lenslets (or referred to as micro lenses)distributed in an array. A propagation direction of a light ray from themonitor is changed by at least one lenslet in the lenslet array, and thelight ray with the changed propagation direction forms an image on aretina of a user after passing through an eye of the user (similar to alens). If a partial image formed on the retina is a relatively smallaverage circle of confusion, it is equivalent to that the user can see aclear virtual image which is displayed in space and corresponds to thepartial image. On the contrary, if a partial image formed on the retinahas a relatively large average circle of confusion, it is equivalent tothat the user sees a vague virtual image which is displayed in space andcorresponds to the partial image. In addition, because the monitor andthe lenslet array are arranged close to each other (wherein it should benoted that, a relative distance between components in the figure merelyprovides an exemplary illustration, and does not represent an actualdistance between the components), the light ray whose propagationdirection has been changed by the at least one lenslet forms a virtualimage on a side, which is away from the user, of the monitor. Thisvirtual image corresponds to the image formed on the retina of the user,and is equivalent to an image seen by a human eye through the lensletarray; the virtual image may present a certain depth distribution inspace, and the distribution of a circle of confusion of the virtualimage in a current first display plane of the light field display devicecorresponds to the distribution of a circle of confusion of the imageformed on the retina. For example, if some part of the virtual image hasa relatively large average circle of confusion in the first displayplane, an image of this part formed on the retina also has a relativelylarge average circle of confusion, and vice versa.

In a light field display process, the content displayed by the lightfield display device has a display DoF range in space, wherein thedisplay DoF range is a range that has a particular depth width in adepth direction, and the range comprises a depth position of the firstdisplay plane; the first display plane is a display plane that iscorresponding to a focusing distance for viewing in the depth directionand perpendicular to the depth direction. The focusing distance forviewing may be a default human eye focusing distance (for example, it isconsidered by default that an optimal distance for viewing a virtualimage formed in the first display plane is 25 cm, and at this viewingdistance, a human eye can clearly see the virtual image formed in thefirst display plane; in other words, a clearly image of the contentdisplayed by the first display plane can be formed on the retina of thehuman eye located at the focusing distance for viewing). Alternatively,the focusing distance for viewing may be an actual human eye focusingdistance, or the like. In an actual application, after a suitablefocusing distance for viewing is determined, the first display planecorresponding to the focusing distance for viewing can be determined. Anear-to-eye light field display device is used as an example fordescription. When the near-to-eye light field display device is used(for example, wearing smart glasses having a light field displaycapability) to watch the displayed content, a distance L between thehuman eye and the near-to-eye light field display device is relativelyfixed. A difference value between the focusing distance for viewing andthe distance L is calculated, and a depth position corresponding to thedifference value is determined in the depth direction, wherein thisdepth position is the depth position of the first display plane. Afterthe first display plane is determined, the display DoF range of thelight field display device may be determined. For example, a permissiblecircle of confusion may be determined according to one or moreparameters of a display quality expectation, a human eye characteristic,and the like; average imaging circles of confusion at differentdistances from the first display plane in the depth direction arecalculated; and a continuous depth range, which comprises the depthposition of the first display plane and of which an average imagingcircle of confusion is less than the permissible circle of confusion, isdetermined as the display DoF range. Further, the display DoF range maybe optimized with reference to the visual sensitivity of a human eye tothe clearness of content displayed at different depths. For example, thehuman eye is more visually sensitive to the clearness of contentdisplayed at a near place than to the clearness of content displayed ata further place. Therefore, with reference to the depth position of thefirst display plane, a permissible circle of confusion at a depthposition close to the depth position of the first display plane may bedetermined as a lower permissible circle of confusion threshold (C1),and a permissible circle of confusion at a depth position away from thedepth position of the first display plane may be determined as an upperpermissible circle of confusion threshold (C2), thereby determining thedisplay DoF range. The determined display DoF range is expressed as aparticular depth range that comprises the depth position of the firstdisplay plane and that is asymmetrically distributed along the depthposition of the first display plane. Certainly, the foregoing method fordetermining the first display plane and the display DoF range is merelyan illustrative description, and a person skilled in the art can alsouse other determining methods, which are not limited in the embodimentsof the present application. A virtual image displayed in the display DoFrange is very clearly for a human eye at the focusing distance forviewing corresponding to the first display plane, while a virtual imagedisplayed outside the display DoF range is very vague for a for a humaneye at the focusing distance for viewing corresponding to the firstdisplay plane.

The content is content being displayed or content to be displayed. Thedepth distribution information of the content means distributioninformation of each part of the content in a depth direction when thecontent is in a displayed state. At least one depth distributionsub-region may be determined according to the depth distributioninformation of the content, and used as a depth distribution sub-regionon which display quality control needs to be performed, wherein eachdepth distribution sub-region in the at least one depth distributionsub-region is located outside the display DoF range of the opticaldisplay device.

S102: Tilt a first lenslet with respect to an original plane of alenslet array of the light field display device according to at leastthe display DoF range of the light field display device and the depthdistribution sub-region, wherein the first lenslet is a lenslet that isin the lenslet array of the light field display device and affectsdisplay of a first object, and the first object is a part, which islocated in the depth distribution sub-region, of the content.

During implementation of the embodiments of the present application, theinventor of the present application finds that, if the clearness ofdifferent parts of the content needs to be adjusted, the current firstdisplay plane of the light field display device may be adjusted, andafter the first display plane is changed, it is equivalent to that thedisplay DoF range of the light field display device is re-determined. Apart clearly displayed by the light field display device can be adjustedbased on the depth distribution information of the displayed content andthe re-determined display DoF range. For example, content that isclearly displayed by the light field display device currently is aforeground part having a relatively small depth, and by adjusting thefirst display plane, the content clearly displayed content can bechanged to be a background part having a relatively large depth, or thelike. To re-determine the first display plane, source content displayedby the light field display device needs to be processed. This requires alarge amount of computation, the computation is highly complex, and thedisplay speed is low. Therefore, the present application provides a newsolution.

The inventor notices that by tilting a lenslet, an imaging distance of afirst object can be changed. Therefore, in the technical solutionprovided by the embodiments of the present application, this imagedistance adjustment means is fully utilized: a first lenslet unit istilted with respect to an original plane of the lenslet array (whereinthe original plane is a plane in which the lenslet array is located whenall lenslets of the lenslet array are in a non-tilted state) at leastaccording to a current display DoF range of the light field displaydevice and a depth distribution sub-region on which image qualitycontrol needs to be performed, to change a display image distance of thefirst object in space by means of control over the tilt of the firstlenslet, so that an average display image distance of a virtual imagewhich is formed after the first object is displayed by the first lensletis distributed in the current display DoF range of the light fielddisplay device as far as possible, thereby improving display quality ofthe first object displayed by the first lenslet. A near-to-eye lightfield display is used as an example for description. Assuming that adistance between the first display plane and a human eye is 25 cm (thatis, the focusing distance for viewing is 25 cm), an image formed in aparticular depth range of the first display plane (that is, the displayDoF range), for example, the depth range of [25 cm-5 cm, 25 cm+7 cm], isclear to the human eye, and if a user wants to see an image outside thedisplay DoF range, for example, if the user wants to clearly see animage in a range of [32 cm, 35 cm] from the human eye (that is, in aparticular depth distribution sub-region outside the display DoF range),a corresponding lenslet may be tilted at least with respect to theoriginal plane of the lenslet array, so that images of a same object aredisplayed in the display DoF range, thereby improving the displayclearness of the object, to allow the user to see the object clearly. Inthis solution, on the basis of clear imaging in an original display DoFrange, a control means of tilting a lenslet is fully utilized to improvedisplay quality of an object in at least one depth distributionsub-region outside the display DoF range, achieving an effect equivalentto that a user can clearly see an object which is in a depth rangelarger than the original display DoF range; moreover, in the displayquality adjustment process, it is unnecessary to perform complexcomputation on source content of a light field, and by adjusting a focallength of a corresponding lenslet, display quality of an object outsidethe original display DoF range can be adjusted. The solution is simpleand easy to control.

The first lenslet is a lenslet that is in the lenslet array of the lightfield display device and affects display of the first object. The firstobject is a part, which is in the depth distribution sub-region, of thecontent, and if the content is regarded as a complete display image, thefirst object is a part of the complete display image. Display of thefirst object may be affected by one or more lenslets. Therefore, in anactual application, there may be one or more first lenslets. Tiltcontrol may be performed on the one or more first lenslets, so as toimprove the clearness of the displayed first object and improve displayquality.

When the first object is displayed by using, for example, the lightfield display device, affected by factors such as the property of anoptical wave and an aberration, imaging light rays of points on theobject usually cannot be converged at one point, but form a diffusionprojection in a shape of a circle or an ellipse or in anothercircle-like shape, that is, a circle of confusion, which is also knownas disk of confusion, circle of indistinctness, blur circle, or blurspot. The size of a circle of confusion for imaging of a point may becalculated according to the following formula:

$\begin{matrix}{c_{a} = {\frac{{d_{a} - d_{0}}}{d_{a}}E}} & (1)\end{matrix}$

In the formula above: E represents a size of a human eye pupil, forexample, a diameter of a human eye pupil; d_(a) represents a focusingdistance for viewing, that is, a distance between a current firstdisplay plane of a light field display device and the human eye pupil;d₀ represents a display depth of an object, for example, an averagedistance, in a depth direction, between a virtual image of the objectand a human eye located at the focusing distance for viewing; c_(a)represents a size of a circle of confusion (for example, a diameter of acircle of confusion) generated during imaging, at the human eye, of animage point having a display depth of d₀.

A circle of confusion distribution, which is in the first display planeof the light field display device, of a virtual image that is formed inspace after the first object is displayed by the tilted first lensletcorresponds to a circle of confusion distribution, which is on theretina, of an image that is formed on the retina after the virtual imageundergoes an equivalent lens effect of the human eye. Usually, if acircle of confusion has a relatively small size, an image of an objectcorresponding to the circle of confusion is relatively clear, and can beregarded as an in-focus image of the point with respect to the humaneye, wherein the image is clear; correspondingly, if a size of a circleof confusion exceeds a particular permissible range, an image of anobject corresponding to the circle of confusion is relatively blur. Inthe embodiments of the present application, the tilt of the firstlenslet is controlled, and a convergence condition for tilt control ofthe first lenslet is as follows: an average circle of confusion of animage, which is formed on the first display plane of the light fielddisplay device or on the retina after a corresponding object isdisplayed by the tilted first lenslet, is less than or equal to apermissible circle of confusion. In other words, by means of controlover the tilt of the first lenslet, an average circle of confusion ofimages, which are formed on the first display plane of the light fielddisplay device or on the retina by points of the first object displayedby the tilted first lenslet, is less than or equal to a permissiblecircle of confusion. As shown in FIG. 2b , an average circle ofconfusion of a retina image, which corresponds to a virtual image of thefirst object displayed by the tilted lenslet, is reduced, and thereforeaverage clearness of points in the first object is improved. It shouldbe noted that, the term “points in the first object” expresses arelative concept, for example, a part displayed by an individual displaypixel in the monitor may be used as a point in the first object, whichis not limited in the embodiments of the present application.

In any technical solution provided in the embodiments of the presentapplication, the determining manner and the number of the depthdistribution sub-regions may be determined according to an actualrequirement, and the implementation manner is very flexible. Forexample, at least a partial depth region of the content outside thedisplay DoF range may be determined as the at least one depthdistribution sub-region according to input information of a user; foranother example, at least a partial depth region of the content outsidethe display DoF range may be determined as the at least one depthdistribution sub-region according to an image analysis result of thedisplay DoF range and the content; and so on. It can be understood that,the determined at least partial depth region, as a whole, may be used asthe depth distribution sub-region, or the determined at least partialdepth region may be divided into multiple depth distribution sub-regionsaccording to a particular condition, and so on. The foregoing specificimplementation manner is not limited in the embodiment of the presentapplication, so that diversified actual application requirements can bebetter satisfied.

A depth range of the depth distribution sub-region and the number ofdepth distribution sub-regions may be flexibly determined in other depthranges, which are outside the display DoF range, of the content, tosatisfy diversified application requirements.

Optionally, at least one of the at least one depth distributionsub-region is continuous in terms of depth with the display DoF range.In other words, one edge of at least one depth distribution sub-regionin the at least one depth distribution sub-region is adjacent to oneedge of the display DoF range in the depth direction, as shown in FIG.3a or FIG. 3b . This solution helps improve display quality of anobject, which is on the periphery of the display DoF range, in thecontent being displayed or the content to be displayed. In terms of aviewing effect of the user, this is equivalent to that the user canclearly see an object in a depth range larger than the original displayDoF range.

Optionally, at least one of the at least one depth distributionsub-region is discontinuous in terms of depth with the display DoFrange. In other words, none of edges of at least one depth distributionsub-region in the at least one depth distribution sub-region is adjacentto any edge of the display DoF range in the depth direction, as shown inFIG. 3b or FIG. 3c . This solution helps improve display quality of anobject, which is outside the display DoF range, in the content beingdisplayed or the content to be displayed. In terms of a viewing effectof the user, this is equivalent to that the user can clearly see anobject in a depth range larger than the original display DoF range.

After the depth distribution sub-region is determined, the first lensletmay be tilted with respect to the original plane of the lenslet array ofthe light field display device according to at least the display DoFrange of the light field display device and the depth distributionsub-region. A specific implementation manner of tilt control is veryflexible.

In an optional implementation manner, the tilting a first lenslet withrespect to an original plane of a lenslet array of the light fielddisplay device according to at least the display DoF range of the lightfield display device and the depth distribution sub-region comprises:determining expected display depth information of the first objectaccording to the display DoF range and the depth distributionsub-region; determining an expected tilt angle of the first lensletaccording to at least a focal length of the first lenslet and theexpected display depth information; and tilting the first lenslet withrespect to the original plane of the lenslet array according to at leastthe expected tilt angle. After the first lenslet is tilted with respectto the original plane of the lenslet array, as shown in FIG. 2b , theplane in which the first lenslet is located is no longer parallel to theplane in which the monitor is located. With reference to the Scheimpflugprinciple, when an extended plane of a plane wherein an image, which isformed by means of the first lenslet, of the first object in the depthdistribution sub-region is located (the plane is referred to as anobject display plane below), an extended plane of a plane in which thetilted first lenslet is located, and an extended plane of a plane inwhich the monitor is located intersect at a straight line (which may bereferred to as a Scheimpflug line), the first object in the depthdistribution sub-region can obtain a maximum clear imaging area and aminimum average circle of confusion; a tilt angle of the first lensletsatisfying the Scheimpflug principle is the expected tilt angle in theembodiment of the present application.

The object display plane may be obtained by means of fitting accordingto an expected distribution, in the display DoF range, of each imagepoint in the depth distribution sub-region. For example, an optionalimplementation manner is:

(a1) Assuming that the content is a clear 2D image (wherein after lightfield display, the 2D image is displayed as a 3D image having aparticular display depth distribution in space), an image coordinate set{(x_(i),y_(i))} that comprises respective coordinates, in the 2D image,of points in the first object in a partial depth distribution sub-regionof the 2D image may be determined according to depth distributioninformation of the 2D image.

(b1) According to the determined image coordinate set of the points, afirst space coordinate set of respective virtual image pointscorresponding to the points is determined, and first space coordinatesof any image point may be expressed as p(x_(i)′,y_(i)″,d_(i)′), whereinx_(i)′=M₁x_(i), y_(i)′=M₂y_(i), is a known scaling coefficient, forexample, M1 is a known X-direction scaling coefficient and M2 is a knownY-direction scaling coefficient, and d_(i)′ represents an image distanceof the image point, that is, an image distance before the image point ismapped.

(c1) According to the determined first space coordinate set and thecurrent display DoF range of the light field display device, coordinatesof each point in the first space coordinate set are mapped into aparticular depth sub-range of the display DoF range respectively,thereby obtaining a second space coordinate set, and second spacecoordinates of any image point may be expressed asp′(x_(i)″,y_(i)″,d_(i)″), wherein x_(i)″ represents a horizontalcoordinate of the image point in the first display plane; y_(i)″represents a vertical coordinate of the image point in the first displayplane; and d_(i)″ represents an image distance of the image point aftermapping. The mapping may be performed according to a particular rule.For example, on the premise of maintaining a relative depth relationshipof image points in the depth distribution sub-region, the image pointsare mapped to a particular depth sub-range in the display DoF range. Inother words, before adjustment, if a relative depth relationship betweenany two image points: image point p₁ and image point p₂, is that imagedistances of the two image points satisfy that d_(p2)≧d_(p1), then theimage distances of the two image points after the adjustment satisfythat d_(p2)′≧d_(p1)′. What is changed before and after the adjustment isa difference value between image distances. For example, a difference(d_(p2)′−d_(p1)′) between the image distances of the two image pointsbefore the adjustment is greater than a difference (d_(p2)′−d_(p1)′)between the image distances of the two image points after theadjustment, that is, (d_(p2)−d_(p1))≧(d_(p2)′−d_(p1)′). This isequivalent to a certain degree of depth compression on the first objectin the depth distribution sub-region, so as to improve display qualityof more parts of the first object. Optionally, proportional mapping maybe performed on the premise of keeping the relative depth relationshipof image points unchanged. For example, if four image points whose imagedistances are {25, 28, 42, 45} respectively need to be mapped into adisplay depth sub-region [30, 40], image distances of the image pointsafter the mapping are {25+(30−25)*25/25, (28+30−25)*25/28, (42−5)*45/42,(45−5)*45/45} respectively, and in this way, the relative depthrelationship of the image points is kept unchanged before and after themapping.

(d1) A plane is obtained by means of fitting according to the secondspace coordinate set. This step is equivalent to providing a point cloudand obtaining a plane by means of fitting according to the point cloud,to make a sum of distances from points in the point cloud to the planeminimum. Further, during the fitting process, it can be ensured that theminimum distance sum is less than a preset value; otherwise, multipleplanes may be obtained by means of fitting until a minimum distance sumis less than the preset value. An algorithm used in plane fittingbelongs to the prior art. For example, a Random Sample Consensus (RANSC)is used, but the algorithm is not limited thereto, and details are notdescribed herein.

Further optionally, the expected tilt angle of the first lenslet may becalculated according to the following formula:

$\begin{matrix}{{\tan \; \varnothing} = {\frac{v}{{v\; \cos \; \theta} - f}\sin \; \theta}} & (3)\end{matrix}$

In the foregoing formula: θ represents a tilt angle of the first lensletwith respect to the original plane of the lenslet array, that is, anexpected tilt angle of the first lenslet; f represents a focal length ofthe first lenslet, Ø represents an angle between the object imagingplane and the plane in which the monitor is located, and ν representsexpected display depth information of the first object. The firstlenslet is tilted with respect to the original plane of the lensletarray according to at least the expected tilt angle determined in thissolution, so that a tilt angle of the first lenslet with respect to theoriginal plane of the lenslet array (for example, an angle between anoptical axis of the tilted first lenslet and the original plane of thelenslet array) is close to the expected tilt angle as much as possibleor even equal to the expected tilt angle. This solution improves tiltcontrol efficiency of the first lenslet, and the first object displayedby the tilted first lenslet can obtain a relatively large clear imagingarea.

In an actual application, the tilt of the first lenslet with respect tothe original plane of the lenslet array may be controlled according tothe expected tilt angle, so that the tilt angle of the first lensletwith respect to the original plane of the lenslet array is close to theexpected tilt angle as much as possible or even equal to the expectedtilt angle.

Alternatively, a tilt direction of the first lenslet may also bedetermined according to a relative distribution of the depthdistribution sub-region and the display DoF range in the depthdirection, and the first lenslet is tilted with respect to the originalplane of the lenslet array according to the tilt direction and theexpected tilt angle. By means of this solution, tilt control efficiencyof the first lenslet can be improved, so that tilt control on the firstlenslet satisfies a convergence condition of the average circle ofconfusion as quickly as possible. Specifically speaking, the tiltedfirst lenslet can change the display image distance of the first object,and if an actual image distance of the first object needs to be adjustedso that it falls in the display DoF range of the light field displaydevice, the tilt direction of the first lenslet may be determinedaccording to the relative distribution of the depth distributionsub-region and the display DoF range in the depth direction, to improvethe tilt control efficiency of the first lenslet. For example, as shownin FIG. 2b , if the depth distribution sub-region of the first object islocated at a relatively deep position with respect to the display DoFrange (for example, an image distance d_(p2) of an image point p₂obtained after a point is displayed by the non-tilted first display unitand the first lenslet corresponding to the first display unit is locatedat a relatively deep position with respect to the display DoF range), itis expected to adjust the display image distance of the first object toa depth position with a relatively small depth value (for example, it isexpected that the image point p₂, which has the image distance d_(p2)when the point is displayed by the non-tilted first lenslet, forms animage at an image distance d_(p2)″ by means of the tilted first lenslet,and d_(p2)>d_(p2)″), and in this case, it may be determined that thetilt direction of the first lenslet is a first direction, wherein thefirst direction is a direction of increasing an angle between the planein which the first lenslet is located and the plane in which the monitoris located. The first lenslet is tilted towards the first direction withrespect to the original plane of the lenslet array by the expected tiltangle, and in this way, the convergence condition of the average circleof confusion can be satisfied as quickly as possible in the tilt controlprocess of the first lenslet, thereby improving the tilt controlefficiency of the first lenslet. For another example, if the depthdistribution sub-region of the first object is located at a relativelyshallow position with respect to the display DoF range, it is expectedto adjust the display image distance of the first object to a depthposition with a relatively large depth value, and in this case, it maybe determined that the tilt direction of the first lenslet is a seconddirection, wherein the second direction is a direction of decreasing anangle between the plane in which the first lenslet is located and theplane in which the monitor is located. The first lenslet is tiltedtowards the second direction with respect to the original plane of thelenslet array by the expected tilt angle, and in this way, theconvergence condition of the average circle of confusion can besatisfied as quickly as possible in the tilt control process of thefirst lenslet, thereby improving the tilt control efficiency of thefirst lenslet.

In another optional implementation manner, the tilting a first lensletwith respect to an original plane of a lenslet array of a light fielddisplay device according to at least a display DoF range of the lightfield display device and the depth distribution sub-region comprises:determining expected display depth information of the first objectaccording to the display DoF range and the depth distributionsub-region; determining a permissible tilt angle range of the lensletaccording to at least a focal length of the first lenslet and theexpected display depth information; and tilting the first lenslet withrespect to the original plane of the lenslet array according to at leastthe permissible tilt angle range. By means of this solution, the firstlenslet is tilted with respect to the original plane of the lensletarray, and an angle between the tilted first lenslet and the originalplane of the lenslet array (for example, an angle between an opticalaxis of the tilted first lenslet and the original plane of the lensletarray) falls in the permissible angle range. The permissible angle rangemay be flexibly determined according to a requirement that the tiltedlenslet can reduce an average circle of confusion in imaging of acorresponding object, which is not limited in the embodiment of thepresent application. This solution improves the tilt control efficiencyof the first lenslet, and can improve the clearness of the first objectdisplayed by the tilted first lenslet.

The permissible angle range may be predetermined according to therequirement that the tilted lenslet can reduce the corresponding averagecircle of confusion, and a determining method may comprise but notlimited to: determining by means of experiment, determining by means ofanalog simulation, determining by means of formula derivation, and thelike. Optionally, the determining a permissible tilt angle range of thelenslet according to at least a focal length of the first lenslet andthe expected display depth information comprises: determining anexpected tilt angle of the first lenslet according to at least the focallength of the first lenslet and the expected display depth information;and determining a permissible angle range according to at least theexpected tilt angle. For a method for determining the expected tiltangle, refer to the description above. After the expected tilt angle isdetermined, with reference to factors such as a tilt adjustmentprecision limitation of the lenslet and a display quality requirement onan object, an angle range of the expected tilt angle within a certainpermissible error range is used as the permissible angle range. Thepermissible angle range determined by using this solution is relativelyreasonable, and control over the tilt of the lenslet with reference tothe imaging plane performed based on the permissible angle range isefficient and easy to implement.

In an actual application, the tilt of the first lenslet with respect tothe original plane of the lenslet array may be controlled according tothe permissible angle range, so that the tilt angle of the first lensletwith respect to the original plane of the lenslet array falls in thepermissible angle range.

Alternatively, a tilt direction of the first lenslet may be determinedaccording to a relative distribution of the depth distributionsub-region and the display DoF range in the depth direction; and thefirst lenslet is tilted with respect to the original plane of thelenslet array according to the tilt direction and the permissible tiltangle range. For a method for determining the tilt direction of thefirst lenslet, refer to the description above. By means of thissolution, the tilt control efficiency of the first lenslet can beimproved, so that tilt control on the first lenslet satisfies aconvergence condition of the average circle of confusion as quickly aspossible.

Further, with reference to any technical solution provided by theembodiments of the present application, the method for determining theexpected display depth information is very flexible, and is not limitedin the embodiments of the present application.

Optionally, the determining expected display depth information of thefirst object according to at least the display DoF range comprises:determining any display depth in the display DoF range as the expecteddisplay depth information of the first object. For example, the displayDoF range of the light field display device is [V_(close), V_(far)], anda display depth in the display DoF range may be determined as theexpected display depth information of the first object, that is,expected display depth information V′ε[V_(close),V_(far)] of each pointin the first object. In an actual application, a preferable displaydepth in the display DoF range may be determined as the expected displaydepth information of the first object according to one or more factorssuch as a focal length adjustment capability of the first lenslet and arelative distribution of objects in different depth distributionsub-regions of the content, and the expected display depth informationis used as a basis for tilt control of the first lenslet, therebyimproving the flexibility of solution implementation.

Optionally, the determining expected display depth information of thefirst object according to at least the display DoF range comprises:determining a display depth, which is in the display DoF range and closeto the depth distribution sub-region, as the expected display depthinformation of the first object according to a relative distribution ofthe depth distribution sub-region and the display DoF range in the depthdirection. The term “close to” is a relative concept taking apermissible range into consideration, and in an actual application, theexpected display depth information may be determined with reference to adepth position V_(focus) of a current display focus plane of the lightfield display device. For example, if the depth distribution sub-regioncorresponding to the first object is located at a relatively deep areawith respect to the display DoF range, a display depth in the range of[V_(focus),V_(far)] (for example, a depth position as close to V_(far)as possible in the range of [V_(focus), V_(far)]) may be determined asthe expected display depth information of the first object; if the depthdistribution sub-region corresponding to the first object is located ata relatively shallow area with respect to the display DoF range, adisplay depth in the range of [V_(focus),V_(far)] (for example, a depthposition as close to V_(close) as possible in the range of[V_(focus),V_(far)]) may be determined as the expected display depthinformation of the first object. By means of such processing, a relativerelationship of display depths between objects, of the content,distributed at different depths can be kept unchanged as far aspossible. An object having a greater depth has a greater display imagedistance than an object having a smaller depth, so that the sense ofdepth when the user watches the image is not reduced while the clearnessof the image is improved, thereby improving user experience.

With reference to any technical solution provided by the embodiments ofthe present application, optionally, before the tilting a first lensletwith respect to an original plane of a lenslet array of a light fielddisplay device, the method further comprises: determining a lenslet,which is in the lenslet array and affects display of the first object,as the first lenslet. By means of this solution, one or more lenslets,which are in the lenslet array of the light field monitor and affectdisplay of the first object, may be determined as the first lenslets,and then tilt control is performed on the one or more first lenslets, tochange the display image distance of the first object displayed by thecorresponding first lenslet, so that the display image distance falls inthe display DoF range of the light field display device, therebyimproving the display quality of the first object.

Further optionally, the determining a lenslet, which is in the lensletarray and affects display of the first object, as the first lensletcomprises: determining a first display sub-region according to theexpected display depth information, the focusing distance for viewing,and a pupil size of a human eye, wherein the first display sub-region isa display sub-region that is in the monitor of the light field displaydevice and affects display of the first object; and determining,according to a mapping relationship between display sub-regions in themonitor and lenslets in the lenslet array, a lenslet which is in thelenslet array and corresponding to the first display sub-region as thefirst lenslet. In some cases of light field display, content displayedon the monitor and a virtual image that a user sees through a lensletarray are the same in terms of content but have different imageexpression forms. The virtual image that the user sees through thelenslet array is usually consistent with an image seen by a human eye ina natural environment, while the monitor displays multiple sub-images,each sub-image corresponds to a local part of the virtual image, andobjects represented by different sub-images may be partially the same.For example, light field display is performed by using triple angularsamples (that is, 3×3 display sub-regions are used to represent displayinformation of a same object from different angles). For example, a sameobject such as a beak may be shown in 9 sub-images, lensletscorresponding to the display sub-regions that display the 9 sub-imagesmay be determined as the first lenslets to be adjusted, and the focallengths of the first lenslets are adjusted to improve the imageclearness of the object. In an actual application, the first displaysub-region may be determined according to at least the depthdistribution information of the content, and then the first lenslet isfurther determined according to the first display sub-region. Forexample, referring to FIG. 2c , determining a first lenslet that affectsimaging of a point of the first object is used as an example to describethe method for determining the first lenslet:

(a2) Assuming that the content is a clear 2D image (wherein after lightfield display, the 2D image is displayed as a 3D image having aparticular display depth distribution in space), image coordinates{(x_(i),y_(i))}, in the 2D image, of points in the first object in apartial depth distribution sub-region of the 2D image may be determinedaccording to depth distribution information of the 2D image. It shouldbe noted that, the determined image coordinates of the points may be ina concentrated distribution or in a dispersed distribution. A point inthe first object is used as an example for description.

(b2) According to the determined image coordinates of the point, spacecoordinates p(x_(i)′,y_(i)′) of a virtual image point P corresponding tothe point are determined, wherein x_(i)′=M₁x_(i), y_(i)=M₂y_(i), M is aknown scaling coefficient, for example, M1 is a known X-directionscaling coefficient and M2 is a known Y-direction scaling coefficient.

(c2) The virtual image point P may be regarded as a virtual lightsource. Entering of a light ray emitted by the virtual light source intoa human eye is related to a distance between the virtual light sourceand the human eye and the pupil size of the human eye, while thedistance between the virtual light source and the human eye isequivalent to a display depth of the virtual image point P, and adiameter of the pupil of the human eye can be acquired in advanceaccording to an empirical value, an experiment value, or a detectedvalue. After the two pieces of information are determined, a light conewith the virtual image point P as a vertex is also determinedcorrespondingly, and light rays in the light cone can enter the humaneye. Therefore, light rays emitted by display sub-regions covered by across section between the monitor and the light cone may be regarded aslight rays capable of entering the human eye, and these displaysub-regions are display sub-regions that affect imaging of the virtualimage point, namely, the first display sub-regions. Optionally, acoverage area of the virtual image point P on the monitor along avertical direction may be determined according to formulas (3) and (4),thereby determining the first display sub-region according to thecoverage area:

$\begin{matrix}{y_{\min} = {{- Y_{i\; 2}} = {\frac{\left( {v - d_{l}} \right)\left( {{0.5\; E} - y_{i}^{\prime}} \right)}{d_{v}} + y_{i}^{\prime} - \frac{\left( {v - d_{l}} \right)E}{d_{v}}}}} & (3) \\{{y_{\max} = {{{- y_{i}^{\prime}} + Y_{i\; 1}} = {\frac{\left( {v - d_{l}} \right)\left( {{0.5\; E} - y_{i}^{\prime}} \right)}{d_{v}} + y_{i}^{\prime}}}}{{wherein}\text{:}}} & (4) \\{\frac{Y}{E} = \frac{v - d_{l}}{d_{v}}} & (5) \\{\frac{Y_{i\; 1}}{{0.5\; E} - y_{i}^{\prime}} = \frac{v - d_{l}}{d_{v}}} & (6) \\{Y_{i\; 2} = {Y - Y_{i\; 1} - y_{i}^{\prime}}} & (7)\end{matrix}$

In the foregoing formulas: represents a vertical coverage area of across section, in a plane wherein the monitor is located, of a lightcone from the virtual image point P, which is used as a virtual lightsource, to the pupil of the human eye; Y_(i1) represents a verticaldistance from an upper boundary of the cross section between the lightcone and the monitor to a plane 2, Y_(i2) represents a vertical distancefrom a lower boundary of the cross section between the light cone andthe monitor to a plane 1, wherein the plane 1 passes through the centerof the monitor and is perpendicular to the plane of the monitor, and theplane 2 is a plane that passes through the virtual image point P and isparallel to the plane 1; y_(min) represents a minimum coordinate valueof a vertical coverage area of the virtual image point P on the monitor;y_(max) represents a maximum coordinate value of the vertical coveragearea of the virtual image point P on the monitor; d_(l) represents adistance between the monitor and the lenslet array; d_(v) represents adistance between the virtual image and the lenslet array; E andrepresents the diameter of the pupil.

(d2) According to a known start position of each display sub-region inthe vertical direction of the monitor, a serial number of a firstdisplay sub-region in the vertical coverage area of the virtual imagepoint P on the monitor is determined. Because a correspondingrelationship between sub-images and lenslets is known, a serial numberof a lenslet covered in the vertical direction can be obtained.

According to the coverage area and Y-coordinate position information ofeach display sub-region in the monitor, display sub-regionscorresponding to the coverage area can be determined. If the coveragearea completely overlaps with vertical position coordinates of thedisplay sub-regions, the display sub-regions at the correspondingvertical coordinates in the coverage area can be conveniently determinedas the first display sub-regions; if an edge part of the coverage areadoes not completely correspond to a complete display sub-region, thefirst display sub-region may be determined by using an approximateprocessing method, for example, a display sub-region partially coveredby the coverage area is determined as the first display sub-region, oronly a display sub-region completely covered by the coverage area is thefirst display sub-region, or a display sub-region whose coordinateoverlapping sub-area with the coverage area satisfies a particular ratiois the first display sub-region. The determining manner is veryflexible.

(e2) Likewise, by using a method similar to (c2) and (d2), a serialnumber of a first display sub-region in a horizontal-direction coveragearea of the virtual image point P on the monitor is determined, therebyobtaining a serial number of a first lenslet covered in the horizontaldirection.

Lenslets corresponding to the determined lenslet serial numbers are thefirst lenslets on which tilt control is going to be performed. In thissolution, the method for determining a lenslet that affects imagecapture in the depth distribution sub-region is simple and easy toimplement.

A person skilled in the art should understand that, in any of theforegoing methods in the specific implementation manners of the presentapplication, sequence numbers of the steps do not mean a correspondingexecution order. The corresponding execution order of the steps shouldbe determined according to functions and internal logic thereof, andshould not be construed as any limitation on implementation processes ofthe specific implementation manners of the present application.

FIG. 4 is a logic block diagram of a first light field display controlapparatus according to an embodiment of the present application. Asshown in FIG. 4, the light field display control apparatus provided inthe embodiment of the present application may comprise: a depthdistribution sub-region determining module 41 and a tilt control module42.

The depth distribution sub-region determining module 41 is configured todetermine at least one depth distribution sub-region of contentaccording to a display DoF range of a light field display device anddepth distribution information of the content, wherein each depthdistribution sub-region of the at least one depth distributionsub-region is located outside the display DoF range.

The tilt control module 42 is configured to tilt a first lenslet withrespect to an original plane of a lenslet array of the light fielddisplay device according to at least the display DoF range of the lightfield display device and the depth distribution sub-region, wherein thefirst lenslet is a lenslet that is in the lenslet array of the lightfield display device and affects display of a first object, and thefirst object is a part, which is located in the depth distributionsub-region, of the content.

In the technical solution provided by the embodiment of the presentapplication, the first lenslet is tilted with respect to the originalplane of the lenslet array (wherein the original plane is a plane inwhich the lenslet array is located when all lenslets of the lensletarray are in a non-tilted state) according to at least a current displayDoF range of the light field display device and a depth distributionsub-region on which image quality control needs to be performed, tochange a display image distance of the first object in space by means ofcontrol over the tilt of the first lenslet, so that an average displayimage distance of a virtual image which is formed after the first objectis displayed by the first lenslet is distributed in the current displayDoF range of the light field display device as far as possible, therebyimproving display quality of the first object displayed by the firstlenslet. In this solution, on the basis of clear imaging in an originaldisplay DoF range, a control means of tilting a lenslet is fullyutilized to improve display quality of an object in at least one depthdistribution sub-region outside the display DoF range, achieving aneffect equivalent to that a user can clearly see an object which is in adepth range larger than the original display DoF range; moreover, in thedisplay quality adjustment process, it is unnecessary to perform complexcomputation on source content of a light field, and by adjusting a focallength of a corresponding lenslet, display quality of an object outsidethe original display DoF range can be adjusted. The solution is simpleand easy to control.

The light field display control apparatus may execute the light fielddisplay control method in application programs that comprise but notlimited to: image presentation, video playback, and the like, to performimage display control. A device manifestation form of the light fielddisplay control apparatus is not limited. For example, the light fielddisplay control apparatus may be an independent component, and thecomponent is in coordination and communication with a light fielddisplay device; or the light field display control apparatus may beintegrated in a light field display device as a functional module, andthe light field display device may comprise but not limited to: anelectronic device having a light field display capability. For example,the light field display device may comprise but not limited to: anear-to-eye light field display device, and the near-to-eye light fielddisplay device may comprise but not limited to: a smart helmet, smartglasses, and the like.

Optionally, an average circle of confusion, in a first display plane ofthe light field display device, of the first object displayed by thetilted first lenslet is less than or equal to a permissible circle ofconfusion, wherein the first display plane is a display plane that iscorresponding to a focusing distance for viewing in a depth directionand perpendicular to the depth direction. In this solution, reduction ofthe average circle of confusion in the first display plane can be usedas a convergence condition for tilt control on the first lenslet,thereby improving the efficiency and pertinence of the tilt control onthe first lenslet.

Optionally, at least one of the at least one depth distributionsub-region is continuous in terms of depth with the display DoF range.This solution helps improve display quality of an object, which is onthe periphery of the display DoF range, in the content being displayedor the content to be displayed. In terms of a viewing effect of theuser, this is equivalent to that the user can clearly see an object in adepth range larger than the original display DoF range.

Optionally, at least one of the at least one depth distributionsub-region is discontinuous in terms of depth with the display DoFrange. This solution helps improve display quality of an object, whichis outside the display DoF range, in the content being displayed or thecontent to be displayed. In terms of a viewing effect of the user, thisis equivalent to that the user can clearly see an object in a depthrange larger than the original display DoF range.

With reference to any technical solution provided by the embodiments ofthe present application, optionally, as shown in FIG. 5, the tiltcontrol module 42 comprises: an expected display depth informationacquiring sub-module 421, an expected tilt angle determining sub-module422, and a first tilt control sub-module 423. The expected display depthinformation acquiring sub-module 421 is configured to determine expecteddisplay depth information of the first object according to the displayDoF range and the depth distribution sub-region; the expected tilt angledetermining sub-module 422 is configured to determine an expected tiltangle of the first lenslet according to at least a focal length of thefirst lenslet and the expected display depth information; the first tiltcontrol sub-module 423 is configured to tilt the first lenslet withrespect to the original plane of the lenslet array according to at leastthe expected tilt angle. This solution improves the tilt controlefficiency of the first lenslet, and the first object displayed by thetilted first lenslet can obtain a relatively large clear imaging area.

Further optionally, the first tilt control sub-module 423 comprises: afirst tilt direction determining unit 4231 and a first tilt control unit4232. The first tilt direction determining unit 4231 is configured todetermine a tilt direction of the first lenslet according to a relativedistribution of the depth distribution sub-region and the display DoFrange in the depth direction; and the first tilt control unit 4232 isconfigured to tilt the first lenslet with respect to the original planeof the lenslet array according to the tilt direction and the expectedtilt angle. By means of this solution, the tilt control efficiency ofthe first lenslet can be improved, so that tilt control on the firstlenslet satisfies the convergence condition of the average circle ofconfusion as quickly as possible.

With reference to any technical solution provided by the embodiments ofthe present application, optionally, the tilt control module 42comprises: an expected display depth information acquiring sub-module421, a permissible tilt angle range determining sub-module 424, and asecond tilt control sub-module 425. The expected display depthinformation acquiring sub-module 421 is configured to determine expecteddisplay depth information of the first object according to the displayDoF range and the depth distribution sub-region; the permissible tiltangle range determining sub-module 424 is configured to determine apermissible tilt angle range of the lenslet according to at least afocal length of the first lenslet and the expected display depthinformation; and the second tilt control sub-module 425 is configured totilt the first lenslet with respect to the original plane of the lensletarray according to at least the permissible tilt angle range. Thissolution improves the tilt control efficiency of the first lenslet, andcan improve the clearness of the first object displayed by the tiltedfirst lenslet.

Further optionally, the permissible tilt angle range determiningsub-module 424 comprises: an expected tilt angle determining unit 4241and a permissible angle range determining unit 4242. The expected tiltangle determining unit 4241 is configured to determine an expected tiltangle of the first lenslet according to at least the focal length of thefirst lenslet and the expected display depth information; and thepermissible angle range determining unit 4242 is configured to determinethe permissible angle range according to at least the expected tiltangle. The permissible angle range determined by using this solution isrelatively reasonable, and control over the tilt of the lenslet withreference to the imaging plane performed based on the permissible anglerange is efficient and easy to implement.

Further optionally, the second tilt control sub-module 425 comprises: asecond tilt direction determining unit 4251 and a second tilt controlunit 4252. The second tilt direction determining unit 4251 is configuredto determine a tilt direction of the first lenslet according to arelative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction; and the second tilt controlunit 4252 is configured to tilt the first lenslet with respect to theoriginal plane of the lenslet array according to the tilt direction andthe permissible tilt angle range. By means of this solution, the tiltcontrol efficiency of the first lenslet can be improved, so that tiltcontrol on the first lenslet satisfies the convergence condition of theaverage circle of confusion as quickly as possible.

Optionally, as shown in FIG. 6, the expected display depth informationacquiring sub-module 421 comprises: a first expected display depthinformation acquiring sub-module 4211. The first expected display depthinformation acquiring sub-module 4211 is configured to determine anydisplay depth in the display DoF range as the expected display depthinformation of the first object. This solution of determining theexpected display depth information is simple, and the implementationmanner is flexible.

Optionally, the expected display depth information acquiring sub-module421 comprises: a second expected display depth information acquiringsub-module 4212. The second expected display depth information acquiringsub-module 4212 is configured to determine a display depth, which is inthe display DoF range and close to the depth distribution sub-region, asthe expected display depth information of the first object according toa relative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction. By means of this solution, theexpected display depth information is determined and subsequent displaycontrol is performed according to the expected display depthinformation, so that the sense of depth of the user during watching isnot reduced while display clearness is improved, thereby improving userexperience.

Optionally, as shown in FIG. 7, the light field capture controlapparatus further comprises: a lenslet determining module 43. Thelenslet determining module 43 is configured to determine a lenslet,which is in the display array and affects display of the first object,as the first lenslet. After the first lenslet to be adjusted isdetermined, tilt control may be performed on the first lenslet toimprove display quality of the first object.

Further optionally, the lenslet determining module 43 comprises: adisplay sub-region determining sub-module 431 and a lenslet determiningsub-module 432. The display sub-region determining sub-module 431 isconfigured to determine a first display sub-region according to a lightfield image corresponding to the content, wherein the first displaysub-region is a display sub-region that is in the monitor of the lightfield display device and affects display of the first object; and thelenslet determining sub-module 432 is configured to determine, accordingto a mapping relationship between display sub-regions in the monitor andlenslets in the lenslet array, a lenslet which is in the lenslet arrayand corresponding to the first display sub-region as the first lenslet.This solution of determining the first lenslet is simple and easy toimplement.

FIG. 8 is a structural block diagram of a light field display controlapparatus according to an embodiment of the present application. Thespecific embodiment of the present application does not limit a specificimplementation manner of the light field display control apparatus 800.As shown in FIG. 8, the light field display control apparatus 800 maycomprise:

a processor 810, a communications interface 820, a memory 830, and acommunications bus 840.

The processor 810, the communications interface 820 and the memory 830communicate with each other by using the communications bus 840.

The communications interface 820 is configured to communicate with adevice having a communications function, an external light source, andthe like.

The processor 810 is configured to execute a program 832, andspecifically may execute related steps in any one of the foregoingembodiments of the light field display control method.

For example, the program 832 may comprise program code, and the programcode comprises a computer operation instruction.

The processor 810 may be a central processing unit (CPU), or anapplication specific integrated circuit (ASIC), or is configured to beone or more integrated circuits for implementing the presentapplication.

The memory 830 is configured to store the program 832. The memory 830may comprise a random access memory (RAM), or may further comprise anon-volatile memory, for example, at least one magnetic disk memory.

For example, in an optional implementation manner, the processor 810 mayexecute the following steps by executing the program 832: determining atleast one depth distribution sub-region of content according to adisplay DoF range of a light field display device and depth distributioninformation of the content, wherein each depth distribution sub-regionof the at least one depth distribution sub-region is located outside thedisplay DoF range; and tilting a first lenslet with respect to anoriginal plane of a lenslet array of the light field display deviceaccording to at least the display DoF range of the light field displaydevice and the depth distribution sub-region, wherein the first lensletis a lenslet that is in the lenslet array of the light field displaydevice and affects display of a first object, and the first object is apart, which is located in the depth distribution sub-region, of thecontent.

In another optional implementation manner, the processor 810 may furtherexecute steps mentioned in any other embodiment above by executing theprogram 832, and details are not described herein again.

For specific implementation of the steps in the program 832, refer tocorresponding descriptions in the corresponding steps, modules,sub-modules, and units in the foregoing embodiments, and details are notdescribed herein again. It may be clearly understood by a person skilledin the art that, for the purpose of convenient and brief description,for a specific working process of the foregoing system, apparatus, andunit, reference may be made to a corresponding process in the foregoingmethod embodiments, and details are not described herein again.

An embodiment of the present application further provides a light fielddisplay device. As shown in FIG. 9, the light field display devicecomprises: a monitor, a lenslet array, and a light field display controlapparatus; the monitor and the lenslet array are arranged close to eachother (that is, a distance between the monitor and the lenslet array isvery small, and is usually less than a maximum focal length of thelenslets in the lenslet array); the lenslet array comprises multipletiltable lenslets distributed in the array, and the light field displaycontrol apparatus is connected to the monitor and the lenslet array, andperforms tilt control on one or more lenslets to improve displayclearness of an object in at least one depth distribution sub-regionoutside the display DoF range.

The light field display control apparatus may be any light field displaycontrol apparatus provided in the embodiments of the presentapplication, and can execute, in application processes that comprise butnot limited to: image presentation, video playback, and the like, anylight field display control method provided in the embodiments of thepresent application, to perform image display control. For a principleand an apparatus structure for the light field display control performedby the light field display device, refer to the description in otherembodiments of the present application, and details are not describedherein again.

In the embodiments of the present application, the serial numbers and/orcorresponding order of the embodiments is merely used for description,and is not intended to represent merits of the embodiments. Thedescriptions on the embodiments have their respective emphasis, and fora part that is not described in detail in a certain embodiment,reference may be made to related description in another embodiment. Forrelated descriptions of implementation principles or processes relatedto the apparatus, device or system embodiments, reference may be made tothe records of the corresponding method embodiments, which are notrepeated herein.

A person of ordinary skill in the art should appreciate that, incombination with the examples described in the embodiments herein, unitsand method steps can be implemented by electronic hardware, or acombination of computer software and electronic hardware. Whether thefunctions are executed by hardware or software depends on specificapplications and design constraint conditions of the technicalsolutions. A person skilled in the art can use different methods toimplement the described functions for every specific application, but itshould not be considered that this implementation goes beyond the scopeof the present application.

When being implemented in the form of a software functional unit andsold or used as a separate product, the functions may be stored in acomputer-readable storage medium. Based on such understanding, thetechnical solutions of the present invention essentially, or the partcontributing to the prior art, or part of the technical solutions may beimplemented in a form of a software product. The computer softwareproduct is stored in a storage medium, and comprises severalinstructions for instructing a computer device (which may be a personalcomputer, a server, a network device, or the like) to execute all orpart of the steps of the method described in each of the embodiments ofthe present application. The aforementioned storage medium comprises:any medium that can store program codes, such as a USB disk, a removablehard disk, a read-only memory (ROM), a random access memory (RAM), amagnetic disk, or an optical disc.

In the embodiments of the apparatus, method and system of the presentapplication, apparently, each component (such as the system, sub-system,module, sub-module, unit, and sub-unit) or each step may be decomposed,combined and/or combined after being decomposed. The decompositionand/or re-combination should be considered as an equivalent solution ofthe present application. Meanwhile, in the description of the specificembodiments of the present application, a feature described and/or shownfor one implementation manner may be used in one or more otherimplementation manners in an identical or similar way, may be combinedwith a feature in another implementation manner, or may replace afeature in another implementation manner.

It should be emphasized that the term “comprise/contain” used in thistext to indicate existence of a feature, element, step or component, anddoes not exclude existence or addition of one or more other features,elements, steps or components.

Finally, it should be noted that: the aforementioned description of theimplementation manners are merely provided for describing the presentapplication, but not intended to limit the present application. A personof ordinary skill in the art can also make many variations and changeswithout departing from the spirit and the scope of the presentapplication. Therefore, all the equivalent technical solutions also fallwithin the scope of the present application, and the patent protectionscope of the present application shall be limited by the claims.

What is claimed is:
 1. A light field display control method, comprising:determining at least one depth distribution sub-region of contentaccording to a display depth of field (DoF) range of a light fielddisplay device and depth distribution information of the content,wherein each depth distribution sub-region of the at least one depthdistribution sub-region is located outside the display DoF range; andtilting a first lenslet with respect to an original plane of a lensletarray of the light field display device according to at least thedisplay DoF range of the light field display device and the depthdistribution sub-region, wherein the first lenslet is a lenslet that isin the lenslet array of the light field display device and affectsdisplay of a first object, and the first object is a part, which islocated in the depth distribution sub-region, of the content.
 2. Themethod of claim 1, wherein an average circle of confusion, in a firstdisplay plane of the light field display device, of the first objectdisplayed by the tilted first lenslet is less than or equal to apermissible circle of confusion, and the first display plane is adisplay plane that is corresponding to a focusing distance for viewingin a depth direction and perpendicular to the depth direction.
 3. Themethod of claim 1, wherein at least one of the at least one depthdistribution sub-region is continuous in terms of depth with the displayDoF range.
 4. The method of claim 1, wherein at least one of the atleast one depth distribution sub-region is discontinuous in terms ofdepth with the display DoF range.
 5. The method of claim 1, wherein thetilting a first lenslet with respect to an original plane of a lensletarray of the light field display device according to at least thedisplay DoF range of the light field display device and the depthdistribution sub-region comprises: determining expected display depthinformation of the first object according to the display DoF range andthe depth distribution sub-region; determining an expected tilt angle ofthe first lenslet according to at least a focal length of the firstlenslet and the expected display depth information; and tilting thefirst lenslet with respect to the original plane of the lenslet arrayaccording to at least the expected tilt angle.
 6. The method of claim 5,wherein the tilting the tilting the first lenslet with respect to theoriginal plane of the lenslet array according to at least the expectedtilt angle comprises: determining a tilt direction of the first lensletaccording to a relative distribution of the depth distributionsub-region and the display DoF range in the depth direction; and tiltingthe first lenslet with respect to the original plane of the lensletarray according to the tilt direction and the expected tilt angle. 7.The method of claim 1, wherein the tilting a first lenslet with respectto an original plane of a lenslet array of the light field displaydevice according to at least the display DoF range of the light fielddisplay device and the depth distribution sub-region comprises:determining expected display depth information of the first objectaccording to the display DoF range and the depth distributionsub-region; determining a permissible tilt angle range of the firstlenslet according to at least the focal length of the first lenslet andthe expected display depth information; and tilting the first lensletwith respect to the original plane of the lenslet array according to atleast the permissible tilt angle range.
 8. The method of claim 7,wherein the determining a permissible tilt angle range of the firstlenslet according to at least the focal length of the first lenslet andthe expected display depth information comprises: determining anexpected tilt angle of the first lenslet according to at least the focallength of the first lenslet and the expected display depth information;and determining a permissible angle range according to at least theexpected tilt angle.
 9. The method of claim 7, wherein the tilting thefirst lenslet with respect to the original plane of the lenslet arrayaccording to at least the permissible tilt angle range comprises:determining a tilt direction of the first lenslet according to arelative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction; and tilting the first lensletwith respect to the original plane of the lenslet array according to thetilt direction and the permissible tilt angle range.
 10. The method ofclaim 5, wherein the determining expected display depth information ofthe first object according to at least the display DoF range comprises:determining any display depth in the display DoF range as the expecteddisplay depth information of the first object.
 11. The method of claim5, wherein the determining expected display depth information of thefirst object according to at least the display DoF range comprises:determining a display depth, which is in the display DoF range and closeto the depth distribution sub-region, as the expected display depthinformation of the first object according to a relative distribution ofthe depth distribution sub-region and the display DoF range in the depthdirection.
 12. The method of claim 1, wherein before the tilting a firstlenslet with respect to an original plane of a lenslet array of a lightfield display device, the method further comprises: determining alenslet which is in the lenslet array and affects display of the firstobject as the first lenslet.
 13. The method of claim 12, wherein thedetermining a lenslet which is in the lenslet array and affects displayof the first object as the first lenslet comprises: determining a firstdisplay sub-region according to the expected display depth information,the focusing distance for viewing, and a pupil size of a human eye,wherein the first display sub-region is a display sub-region that is inthe monitor of the light field display device and affects display of thefirst object; and determining, according to a mapping relationshipbetween display sub-regions in the monitor and lenslets in the lensletarray, a lenslet which is in the lenslet array and corresponding to thefirst display sub-region as the first lenslet.
 14. A light field displaycontrol apparatus, comprising: a depth distribution sub-regiondetermining module, configured to determine at least one depthdistribution sub-region of content according to a display depth of field(DoF) range of a light field display device and depth distributioninformation of the content, wherein each depth distribution sub-regionof the at least one depth distribution sub-region is located outside thedisplay DoF range; and a tilt control module, configured to tilt a firstlenslet with respect to an original plane of a lenslet array of thelight field display device according to at least the display DoF rangeof the light field display device and the depth distribution sub-region,wherein the first lenslet is a lenslet that is in the lenslet array ofthe light field display device and affects display of a first object,and the first object is a part, which is located in the depthdistribution sub-region, of the content.
 15. The apparatus of claim 14,wherein an average circle of confusion, in a first display plane of thelight field display device, of the first object displayed by the tiltedfirst lenslet is less than or equal to a permissible circle ofconfusion, and the first display plane is a display plane that iscorresponding to a focusing distance for viewing in a depth directionand perpendicular to the depth direction.
 16. The apparatus of claim 14,wherein at least one of the at least one depth distribution sub-regionis continuous in terms of depth with the display DoF range.
 17. Theapparatus of claim 14, wherein at least one of the at least one depthdistribution sub-region is discontinuous in terms of depth with thedisplay DoF range.
 18. The apparatus of claim 14, wherein the tiltcontrol module comprises: an expected display depth informationacquiring sub-module, configured to determine expected display depthinformation of the first object according to the display DoF range andthe depth distribution sub-region; an expected tilt angle determiningsub-module, configured to determine an expected tilt angle of the firstlenslet according to at least a focal length of the first lenslet andthe expected display depth information; and a first tilt controlsub-module, configured to tilt the first lenslet with respect to theoriginal plane of the lenslet array according to at least the expectedtilt angle.
 19. The apparatus of claim 18, wherein the first tiltcontrol sub-module comprises: a first tilt direction determining unit,configured to determine a tilt direction of the first lenslet accordingto a relative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction; and a first tilt control unit,configured to tilt the first lenslet with respect to the original planeof the lenslet array according to the tilt direction and the expectedtilt angle.
 20. The apparatus of claim 14, wherein the tilt controlmodule comprises: an expected display depth information acquiringsub-module, configured to determine expected display depth informationof the first object according to the display DoF range and the depthdistribution sub-region; a permissible tilt angle range determiningsub-module, configured to determine a permissible tilt angle range ofthe lenslet according to at least a focal length of the first lensletand the expected display depth information; and a second tilt controlsub-module, configured to tilt the first lenslet with respect to theoriginal plane of the lenslet array according to at least thepermissible tilt angle range.
 21. The apparatus of claim 20, wherein thepermissible tilt angle range determining sub-module comprises: anexpected tilt angle determining unit, configured to determine anexpected tilt angle of the first lenslet according to at least the focallength of the first lenslet and the expected display depth information;and a permissible angle range determining unit, configured to determinea permissible angle range according to at least the expected tilt angle.22. The apparatus of claim 20, wherein the second tilt controlsub-module comprises: a second tilt direction determining unit,configured to determine a tilt direction of the first lenslet accordingto a relative distribution of the depth distribution sub-region and thedisplay DoF range in the depth direction; and a second tilt controlunit, configured to tilt the first lenslet with respect to the originalplane of the lenslet array according to the tilt direction and thepermissible tilt angle range.
 23. The apparatus of claim 18, wherein theexpected display depth information acquiring sub-module comprises: afirst expected display depth information acquiring sub-module,configured to determine any display depth in the display DoF range asthe expected display depth information of the first object.
 24. Theapparatus of claim 18, wherein the expected display depth informationacquiring sub-module comprises: a second expected display depthinformation acquiring sub-module, configured to determine a displaydepth, which is in the display DoF range and close to the depthdistribution sub-region, as the expected display depth information ofthe first object according to a relative distribution of the depthdistribution sub-region and the display DoF range in the depthdirection.
 25. The apparatus of claim 14, wherein the apparatus furthercomprises: a lenslet determining module, configured to determine alenslet, which is in the lenslet array and affects display of the firstobject, as the first lenslet.
 26. The apparatus of claim 25, wherein thelenslet determining module comprises: a display sub-region determiningsub-module, configured to determine a first display sub-region accordingto the expected display depth information, the focusing distance forviewing, and a pupil size of a human eye, wherein the first displaysub-region is a display sub-region that is in the monitor of the lightfield display device and affects display of the first object; and alenslet determining sub-module, configured to determine, according to amapping relationship between display sub-regions in the monitor andlenslets in the lenslet array, a lenslet which is in the lenslet arrayand corresponding to the first display sub-region as the first lenslet.27. A light field display control apparatus, comprising: a processor, acommunications interface, a memory, and a communications bus, whereinthe processor, the communications interface, and the memory communicatewith each other by means of the communications bus; the memory isconfigured to store at least one instruction, and the instruction causesthe processor to perform the following operations: determining at leastone depth distribution sub-region of content according to a displaydepth of field (DoF) range of a light field display device and depthdistribution information of the content, wherein each depth distributionsub-region of the at least one depth distribution sub-region is locatedoutside the display DoF range; and tilting a first lenslet with respectto an original plane of a lenslet array of the light field displaydevice according to at least the display DoF range of the light fielddisplay device and the depth distribution sub-region, wherein the firstlenslet is a lenslet that is in the lenslet array of the light fielddisplay device and affects display of a first object, and the firstobject is a part, which is located in the depth distribution sub-region,of the content.
 28. A light field display device, comprising: a monitor;a lenslet array, wherein the lenslet array comprises multiple tiltablelenslets distributed in the array; and a light field display controlapparatus of claim 14, wherein the light field display control apparatusis connected to the monitor and the lenslet array.